Fast Stochastic Ordinal Embedding With Variance Reduction and Adaptive Step Size

Learning representation from relative similarity comparisons, often called ordinal embedding, gains rising attention in recent years. Most of the existing methods are based on semi-definite programming (SDP), which is generally time-consuming and degrades scalability, especially confronting large-scale data. To overcome this challenge, we propose a stochastic algorithm SVRG-SBB, has following features: i) achieving good scalability via dropping positive (PSD) constraints as serving fast algorithm, i.e., variance reduced gradient (SVRG) method, ii) adaptive learning introducing new, step size stabilized Barzilai-Borwein (SBB) size. Theoretically, under some natural assumptions, show O (1/T) O(1T) rate convergence to stationary point proposed where T number total iterations. Under further Polyak-?ojasiewicz assumption, can global linear (i.e., exponentially converging optimum) algorithm. Numerous simulations real-world data experiments conducted effectiveness by comparing with state-of-the-art methods, notably, much lower computational cost prediction performance.